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ABSTRACT 

Noting that various types of evaluations of after-school 
programming conducted over the last several years have provided useful 
information to providers and to policymakers, this report summarizes the 
lessons learned from independent evaluations of after-school programs. The 
following overall findings are supported with a delineation of findings from 
various independent evaluations: (1) after-school programs help students 

achieve in school; (2) after-school programs keep children safe; and (3) 
after-school programs help working parents. Additional detail is provided on 
the methodology and findings cited from several, more extensive evaluations 
of the following programs: (1) Los Angeles' Better Education Students for 

Tomorrow; (2) New York City's Beacons Initiative; (3) California Afterschool 
Learning and Safe Neighborhoods Partnerships Program; (4) The After-School 
Corporation [New York City]; (5) Houston's After-School Achievement Program; 
(6) San Diego's "6 to 6" Extended School Day Program; (7) Ohio Urban School 
Initiative School Age Child Care Project; (8) YS-Care After School Program 
for California Work Opportunity and Responsibility to Kids; (9) Massachusetts 
After-School and Other Out-of-School Time Grant Programs; (10) the 
Foundations After-School Enrichment Program; and (11) the Extended-Service 
Schools Initiative. Findings from the MOST Initiative and the Extended- 
Service Schools Initiative are highlighted to illustrate barriers faced by 
after-school programs in obtaining adequate funding, issues related to 
physical space, involvement of the most disadvantaged children, and 
transportation availability. References are included within the text of the 
report. (KB) 
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Although afterschool programs for children have been operating for many years in some 
communities, the afterschool movement - the great national awakening to the opportunity 
~ afterschool offers - is just a few years old. As public demand for afterschool grows, so has the 
o' demand for accountability. That is particularly true in afterschool programs that spend public 
oo dollars. After all, where tax dollars flow, so must accountability to taxpayers. 

Q 

w The Landscape of Afterschool Evaluations 



A number of different types of evaluations have been conducted over the last several years, 
assessing various aspects of afterschool programming. Some evaluations seek to gather data on 
whether programs have been structured as they were originally intended, how well they have 
done at meeting attendance and staffing goals, how they “fit” in the school environment and 
more. Others explore the effect afterschool programs have on the children who participate in 
them, their parents, and even the communities at large. 



Both types of evaluations are of great value to afterschool providers and to policymakers, and 
when taken together the two types of studies help identify the particular program elements and 
approaches most critical to accomplishing program goals. It is useful, for example, to correlate 
information on student attendance at afterschool programs with student academic performance. 
Were an evaluator to conclude that attendance is key to academic gains, program designers 
might focus more energy on improving attendance for students. 




Evaluations also differ by virtue of who conducts them. Many programs self-evaluate, providing 
useful data and satisfying the needs of their various stakeholders - parents, funders, partnering 
businesses, local public officials and so on. But for academics and large funders - the federal 
government, state governments, the Open Society Institute, the Charles Stewart Mott Foundation 
or the Dewitt Wallace- Reader’s Digest Fund, for example - more exacting standards and greater 
independence is often required. It is these independent evaluations that are the subject of this 
document. 




Several studies are discussed in some detail on the pages that follow. 



Summary Lessons from the Data 




The data and conclusions from these studies amply demonstrate the value of individual 
afterschool initiatives. (Citations for the following are included in the detailed descriptions of 
afterschool studies that follow.) 
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Afterschool programs help kids achieve in school. 

> Evaluations of LA’s BEST show that program students’ attendance improved once they 
began participating in the program. That improved attendance led to higher academic 
achievement on standardized tests of math, reading and language arts. In addition, language 
redesignation rates favored LA’s BEST students when compared with non-LA’s BEST 
students. 

> A statewide evaluation of California’s After School Learning and Safe Neighborhoods 
Partnerships Program (ASLSNPP) by the University of California at Irvine demonstrated 
gains closely related to individual students’ level of participation in the program: “Among 
students who participated for more than 150 days (approximately 7.5 months), there was an 
increase of 4.9 percent in students above the 25 th percentile, an increase considerably larger 
than that found statewide. The scores suggest that, particularly for students who participate 
in the ASLSNPP for substantial periods of time, there is a closing of the gap in Math 
achievement between low-income and other students .” [Emphasis in original.] 

> Children in the Ohio Urban School Initiative School Age Child Care Project (SACC) scored 
higher than non-participating students across the state. “SACC 4th grade students’ scores 
exceeded the statewide percentages of students meeting proficiency standards in every 
subject area tested: writing, reading, mathematics, citizenship, and science. SACC 6th 
graders exceeded the statewide percentages of students meeting proficiency standards in four 
of the five areas: writing, reading mathematics, and citizenship.” 

> Reading scores for San Diego’s “6 to 6” students improved. Fifty-seven percent of students 
increased their reading scores over the course of the studied year, and SAT-9 reading scores 
increased. Nearly ten percent of children moved up into the 25 th percentile or higher in 2000 
by comparison to 1 999 reading scores. 

> Forty-four percent of students in San Diego’s “6 to 6” program increased their SAT-9 math 
scores. 

> In Los Angeles’ YS-CARE program, aimed at children from families on TANF, students’ 
reading and mathematics gains outpaced those of non-participating students, as measured by 
SAT-9 scores. 

> Policy Studies Associates’ evaluation of The Afterschool Corporation (TASC) program 
“found significant differences in proficiency-level shifts among active participants and 
nonparticipants who scored in the lowest proficiency level on the 1998-99 mathematics tests. 
In math, 31 percent of active participants scoring at the lowest proficiency level in 1998-99 
scored at a higher proficiency level in 1999-2000, compared to 23 percent of nonparticipants 
who demonstrated the same improvement. Two percent of these active participants increased 
their performance to grade level, compared to 1 percent of nonparticipants. A similar but less 
pronounced pattern was observed on the reading tests administered in grades 3-8. Among 
those scoring at the lowest proficiency level in 1998-99, 45 percent of active participants 
improved their scores in 1999-2000 enough to move to a higher performance level, and 3 
percent scored at grade level. Forty percent of nonparticipants who scored at the lowest 
proficiency level in 1998-99 increased their scores enough to move to a higher proficiency 
level a year later, and 2 percent reached grade level.” 

> Policy Studies Associates’ study of TASC’s third year of operation concluded: “Students 
who participated in TASC after-school activities the most consistently and for the longest 
period of time experienced the greatest math gains, when compared to similar 
nonparticipants. Among students who participated actively in TASC projects in each year of 
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their enrollment, students participating for two years gained an average of four scale -score 
points more on the city-wide standardized tests than similar nonparticipants. Among active 
participants, students participating for three years gained six points more than similar 
nonparticipants. Demonstrating the value of even higher levels of participation, students 
classified as ‘highly active’ (participating 80 percent or more of the days they were enrolled 
in the year and at least 80 days) gained six scale-score points more than similar 
nonparticipants after only two years of TASC participation.” 



Afterschool programs keep kids safe. 

> The LA’s BEST evaluation found that parents and children alike found the safety of the 
afterschool program far superior to the safety within the neighborhood. 

> The evaluation of the New York City Beacons program concluded that “the vast majority of 
youth (85 percent) reported that it was ‘always true’ or ‘mostly true’ that they felt safe at the 
Beacons.” 

> Eighty percent of New York Beacons students who took part in intercept interviews 
described the Beacon as either “very helpful” or “pretty helpful” in helping them avoid drug 
use. 

> Seventy- four percent of New York Beacons students interviewed said that the Beacon was 
either “very helpful” or “pretty helpful” in helping them avoid fighting. 

> In Ohio’s SACC program, “school absence and tardiness were reduced for participating 
students. First graders who were not in a SACC program during kindergarten reduced the 
number of school days they missed from an average of eight during their kindergarten year to 
an average of three days during their 1998-99 1st grade year. Eighth graders who were not in 
a SACC program during 7th grade reduced the average number of school days missed from 
18 to five.” Similarly, “suspensions and expulsions, when comparing the 1998-99 school 
year to the prior school year, were reduced for both elementary school students and middle 
school students who participated in SACC programs.” 

> The TASC evaluation concluded that “staff, students, and parents provided examples of 
student improvements that they attributed to the after-school program. Among the most 
common were improvements in students’ social skills, including the ability to maintain self- 
control, make constructive choices about their behavior, and avoid fights.” 

Afterschool programs help working parents. 

> The LA’s BEST evaluation found that “three-quarters of the parents [surveyed] indicated that 
they worried significantly less about their children’s safety and that they had more energy in 
the evening since enrolling their children in the program. A clear majority also indicated that 
the program resulted in sizeable savings in their time. 

> Parents in the TASC study said that the program helped them balance work and family life: 
94 percent said the program was convenient; 60 percent said they missed less work than 
before because of the program; 59 percent said it supported them in keeping their job; and 54 
percent said it allowed them to work more hours. 

> In Ohio’s SACC program, “the adults in the participating children’s families had a greater 
awareness of community agencies, their facilities, and their services because of the SACC 
programs.” Similarly, “parents participating in interviews or completing surveys felt the 
programs had positive impacts on their families.” 
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Afterschool Evaluations in Detail 



A number of important afterschool evaluations have been conducted - more than enough to 
demonstrate that afterschool programs help children achieve, keep them safe, and help their 
working parents solve the difficult puzzle of child care. Following are summaries of several of 
the most extensive evaluations. 

LA’s BEST 

Los Angeles’s Better Educated Students for Tomorrow, or LA’s BEST, is among the largest and 
best known afterschool programs in the nation. Launched in 1 988 as a partnership between the 
Los Angeles Unified School District (LAUSD), the city of Los Angeles and the private sector, 
the program operates at 104 elementary schools, serving more than 18,000 students. Schools are 
chosen for participation because of the generally low academic achievement among their 
students, or because of the low economic status of the community, or high gang or crime rates in 
the neighborhood. 

Since early in the life of the program, the UCLA Center for the Study of Evaluation has 
conducted evaluation studies. The studies have focused on a variety of topics, using a range of 
measures. The Center released its separate studies in March 1990, March 1991, July 1991, 
December 1993 and spring 1995. Then in June 2000, the Center released a comprehensive 
report summarizing each of the five previous studies and adding a rich set of findings based on 
its five-year tracking of the academic performance and school attendance of LA’s BEST students 
who were in 2 nd through 5 th grades in the 1993-94 school year. 

The study’s description of its methodology: “To study LA’s BEST schools, we obtained 
information about students including ethnicity, gender, language proficiency status, eligibility for 
free/reduced lunch (the proxy for low-income level) and disability status. In addition, we 
collected outcome data including achievement test scores (using either the Comprehensive Test 
of Basic Skills [CTBS] or the more recently adopted Stanford-9 Achievement Test [SAT-9] in 
reading, mathematics, and language arts. The rate at which students were redesignated as fully 
proficient in English was also collected. We also obtained school absence rates, course-taking 
patterns and rates of student mobility (moving between schools or out of the district).” 

In addition, because of the size of the LA’s BEST program and of the LAUSD school system, 
researchers were able to track an extraordinarily large sample of students and a correspondingly 
large “control” group — more than 4,000 LA’s BEST students and more than 15,000 non- 
participating students. The sheer numbers of students tracked make the data produced highly 
reliable. 

The findings, summarized at http://www.lasbest.org/leam/eval.html with a link to the complete 
study, are powerful evidence of the value of afterschool programming. In short, the study found 
that LA’s BEST participants, defined as students who participated regularly and over a period of 
more than one year, when compared to non-participating students, were absent less from school, 
“show positive achievement on standardized tests in mathematics, reading and language arts,” 
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and had “higher language redesignation rates to English proficiency.” [Quoting from LA’s 
BEST’S summary of the findings, at http://www.lasbest.Org/leam/eval.html .1 Specific findings: 

> “[0]ur results show that higher levels of participation in LA’s BEST led to better subsequent 
school attendance, which in turn related to higher academic achievement on standardized 
tests of mathematics, reading and language arts.” [A Decade of Results: The Impact of the 
LA 's BEST After School Enrichment Program on Subsequent Student Achievement and 
Performance, a longitudinal study report and a synthesis of research begun in 1990 by the 
UCLA Center for the Study of Evaluation with support from the Bandai Foundation and the 
City of Los Angeles. June 2000. Denise Huang, Barry Gribbons, Kyung Sung Kim, 
Charlotte Lee, Eva L. Baker. Page 7.] 

> “Language redesignation rates favored LA’s BEST students when compared with non-LA’s 
BEST students for the cohort analyzed (fourth grade, 1994-1995). Significant differences in 
favor of LA’s BEST students were found for subsequent redesignation rates in Grades 6 and 
8. No significant differences were found in comparing performance for Grades 5 and 7.” 
[Decade of Results at page 8.] 

> “Absence follow-up data for the fifth-grade cohort (1994-1995) showed that students who 
participated in LA’s BEST had significantly fewer absences in Grades 6 and 7, although no 
differences were detected in Grades 8 and 9.” [Decade of Results at page 8.] 

> “Although in the initial year LA’s BEST students began with statistically significant 
mathematics achievement scores lower than those of non-participants, in 1997-1998 those 
differences no longer existed.” [Decade of Results at page 9.] 

> “[T]hree-quarters of the parents [surveyed] indicated that they worried significantly less 
about their children’s safety and that they had more energy in the evening since enrolling 
their children in the program. A clear majority also indicated that the program resulted in 
sizeable savings in their time. [Decade of Results at page 14.] 

> “Across the board, parents and children alike found the safety of the afterschool program far 
superior to the safety within the neighborhood.” [Decade of Results, page 14.] 

Researchers conclude: “From our perspective, it looks as if LA’s BEST is a program that, when 
followed as a regular part of students’ broad educational experience, results in statistically 
important differences in student outcomes. The fact that we can detect any change on 
standardized achievement measures in itself is notable, for most educational interventions are 
unable to show impact on measures not tightly tied to the curriculum, or on follow-up 
achievement after a particular program is over. On a practical level, LA’s BEST needs to focus 
its attention on increasing the attendance of enrolled students. It may be that high-level attenders 
do so because they and their parents are more highly motivated, and this interest transfers to 
achievement. But it is equally likely that coming to school and to the LA’s BEST program 
regularly is the reason for good performance and persisting impact subsequent to leaving LA’s 
BEST.” [Decade of Results at pages 9-10.] 

The New York City Beacons Initiative 

Launched in 1991, the Beacons Initiative seeks to link community-based organizations and 
schools in service of children. Programs offer activities for children and families, and seek to 
function as community resources for parents and seniors, as well. The model has been 
duplicated in many cities around the nation. 
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The New York program is the subject of a two-phase evaluation by the Academy for Educational 
Development, the Hunter College Center on AIDS, Drugs and Community Health, and the 
Chapin Hall Center for Children at the University of Chicago. The first phase examined program 
operations in 1997-98; the second, not yet published, will deal with 1998-2000, exploring similar 
issues in greater depth. The evaluation is funded by the Annie E. Casey Foundation, the Open 
Society Institute and the Ford Foundation. Evaluators stress that the study is not specifically 
aimed at documenting student outcomes, although some material of that sort may be useful to 
advocates, particularly from the second phase research. The evaluators’ summary report is 
available online at http://www.aed.org/news/beacons.html . 

Phase One findings include: 

> “The Beacons have clearly created a safe place for youth: the vast majority of youth (85 
percent) reported that it was ‘always true’ or ‘mostly true’ that they felt safe at the Beacons.” 
[Evaluation of the New York City Beacons, Summary of Phase I Findings, 
http://www.aed.org/news/beacons.htm 1 .1 

> “Close to three-fifths of Beacons (57 percent) involve young people in a diverse array of 
community service activities, at least once monthly.” [Evaluation.] 

> “Four- fifths of youth (80 percent) who took part in intercept interviews described the Beacon 
as either ‘very helpful’ or ‘pretty helpful’ in helping them avoid drug use.” [ Evaluation .] 

> “Three-quarters (74 percent) of youth interviewed said that the Beacon was either very 
helpful or pretty helpful in helping them avoid fighting.” [ Evaluation .] 

The California Afterschool Learning and Safe Neighborhoods Partnerships Program 

Begun in 1998, California’s Afterschool Learning and Safe Neighborhoods Partnership Program 
(ASLSNPP) provides $117 million annually in matching funds to local partnerships of school 
districts, community groups, and local governments to provide before and afterschool programs 
for students. In selecting grantees, the state gives priority to schools where 50 percent or more of 
pupils are eligible for free or reduced-cost meals. Programs are designed locally, but are 
required to include an “educational and literacy component to provide tutoring or homework 
assistance in one or more of the following subject areas: language arts, mathematics, history and 
social science, or science”; and “an educational enrichment component, which may include but is 
not limited to, recreation and prevention activities. Such activities might involve the arts, music, 
physical activity, health promotion, and general recreation; work preparation activities; 
community service-learning; and other youth development activities based on student needs and 
interests.” [“California’s Before and After School Learning and Safe Neighborhoods 
Partnerships Program Fact Sheet,” at http://www.cde.ca.gov/afterschool/asDfactsheetapr02.doc. 
September 3, 2002.] In all, 947 afterschool programs were funded during the ASLSNPP’s first 
two years. 

Working with the California Department of Education, the Education Department of the 
University of California at Irvine conducted evaluations of two academic years of the program, 
from 1999 to 2001, releasing results in February 2002. The evaluation relied on data supplied to 
the state by participating programs, as required by law, and examined student and parent 
satisfaction with their programs, as well as students’ academic outcomes. 
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Findings included: 



> SAT-9 scores of participating students increased faster than those of students statewide. In 
reading, 4.2 percent of afterschool students moved from out of the lowest 25 percent of their 
classes. “This increase is more than twice the increase found among all students statewide 
(1.9 percent). . [“Evaluation of California’s After School Learning and Safe Neighborhoods 
Partnerships Program,” Department of Education, University of California at Irvine, 

February 1, 2002, page 4.] In math, similar findings: 2.5 percent of afterschool participants 
moved out of the lowest quartile, compared with 1 .9 percent statewide. [“Evaluation,” page 
6 .] 

> Significantly, gains were closely related to individual students’ levels of participation in the 
program. “Among students who participated for more than 150 days (approximately 7.5 
months), there was an increase of 4.9 percent in students above the 25 th percentile, an 
increase considerably larger than that found statewide. The scores suggest that, particularly 
for students who participate in the ASLSNPP for substantial periods of time, there is a 
closing of the gap in Math achievement between low-income and other students. ” 
[“Evaluation,” page 6. Emphasis in original.] 

> “The regular school day attendance of students in the ASLSNPP increased between 1999 and 
2000. Among the ASLSNPP participants who were absent 5 percent or more days in 1999, 
the average increase in attendance was 5.6 days. Among those who were absent 10 percent 
or more days in 1999, the average increase in attendance was 1 1 days. For those absent 15 
percent or more days in 1999, the average increase in attendance was 17 days.” [Harvard 
Family Research Project, Summary of ASLSNPP Evaluation, 

http://www.gse.harvard.edu/~hfrD/proiects/afterschool/mott/aslsnpp.pdf, September 3, 2002.] 
The After-School Corporation 

The After-School Corporation (TASC) is a New York City-based nonprofit, established by the 
Open Society Institute in 1998, representing a $25 million five-year commitment by the 
foundation. TASC provides grants to nonprofit organizations to establish partnerships with 
individual public schools, and the resulting afterschool programs follow a core set of program 
components. In all, 143 public schools in New York City and 73 schools in other parts of New 
York State participate. Funding is based on enrollment and is $1,000 per student, excluding 
start-up, facilities and staff training. 

The Charles Stewart Mott Foundation and the Carnegie Corporation provided funding to the 
Washington-based Policy Studies Associates to conduct a five-year evaluation, including annual 
summary reports. The first year’s evaluation, covering the 1998-99 school year, focused largely 
on issues related to program design and participation. Among its findings: 

> “Parents’ reasons for enrolling their children in the TASC after-school program include (1) 
making sure their children have a safe place to go after school and (2) wanting to help their 
children perform better in school, which tends to include obtaining extra help for their 
children in reading and math and with daily homework.” [Increasing and Improving After- 
School Opportunities: Executive Summary, 
http://www.tascorp.org/pages/promising esl.pdf, page 13.] 
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> “According to site coordinators, the most challenging areas of implementation included (1) 
hiring qualified staff, (2) recruiting and enrolling students, (3) encouraging high levels of 
student attendance and tracking student participation, (4) developing schedules and curricula, 
(5) building relationships with the school and school staff, (6) building staff cohesiveness, (7) 
establishing policies for project operations, (8) managing fiscal operations, and (9) reaching 
out to families and communities.” [Increasing and Improving After-School Opportunities: 
Executive Summary, http://www.tascorp.org/pages/Dromising esl .pdf , page 13.] 

The second year evaluation, covering the 1999-2000 year, found: 

> “Students reported feeling safe, relaxed, happy, and connected to their after-school program. 
When asked what they especially liked, they mentioned being with their friends, completing 
their homework before going home, and participating in activities that differed from those of 
the regular school day.” [Building Quality and Supporting Expansion of After-school 
Projects, Summary of Findings, ” http://www.tascorp.org/pages/promising es2.pdf, page 12.] 

> “[S]taff, students, and parents provided examples of student improvements that they 
attributed to the after-school program. Among the most common were improvements in 
students’ social skills, including the ability to maintain self-control, make constructive 
choices about their behavior, and avoid fights.” [Building Quality, page 12.] 

> The evaluation “found significant differences in proficiency-level shifts among active 
participants and nonparticipants who scored in the lowest proficiency level on the 1998-99 
mathematics tests. In math, 31 percent of active participants scoring at the lowest proficiency 
level in 1998-99 scored at a higher proficiency level in 1999-2000, compared to 23 percent of 
nonparticipants who demonstrated the same improvement. Two percent of these active 
participants increased their performance to grade level, compared to 1 percent of 
nonparticipants. A similar but less pronounced pattern was observed on the reading tests 
administered in grades 3-8. Among those scoring at the lowest proficiency level in 1998-99, 
45 percent of active participants improved their scores in 1999-2000 enough to move to a 
higher performance level, and 3 percent scored at grade level. Forty percent of 
nonparticipants who scored at the lowest proficiency level in 1998-99 increased their scores 
enough to move to a higher proficiency level a year later, and 2 percent reached grade level.” 
[Patterns of Student-Level Change Linked to TASC Participation Based on TASC Projects in 
Year 2, Executive Summary, 

http://www.policvstudies.com/studies/vouth/Y2%20Performance%20Executive%20Summar 
v.pdf , page 4.] 

P “Forty-five percent of principals in Year 2 reported that the TASC project has increased 
parents’ attendance at school events and 36 percent said that the project had increased 
parents’ attendance at parent-teacher conferences.” Ninety-seven percent of parents 
surveyed indicated that “their child liked to come to the program”; 86 percent agreed “that 
the project was helping their child academically.” Parents also said that the program helped 
them balance work and family life: 94 percent said the program was convenient; 60 percent 
said they missed less work than before because of the program; 59 percent said it supported 
them in keeping their job; and 54 percent said it was supportive to them in allowing them to 
work more hours. [Building Quality, page 15.] 

The third year of the evaluation, covering the 2000-2001 school year, concluded: 
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> “Students who were active participants in TASC projects for more than a year showed 
significantly greater gains on citywide math tests than did similar nonparticipating 
classmates. Students who participated in TASC after-school activities the most consistently 
and for the longest period of time experienced the greatest math gains, when compared to 
similar nonparticipants. Among students who participated actively in TASC projects in each 
year of their enrollment, students participating for two years gained an average of four scale - 
score points more on the city-wide standardized tests than similar nonparticipants. Among 
active participants, students participating for three years gained six points more than similar 
nonparticipants. Demonstrating the value of even higher levels of participation, students 
classified as ‘highly active’ (participating 80 percent or more of the days they were enrolled 
in the year and at least 80 days) gained six scale-score points more than similar 
nonparticipants after only two years of TASC participation. The performance of TASC 
participants on the citywide tests of reading and English/language arts was not significantly 
different from that of similar nonparticipants.” [What Have We Learned from TASC's First 
Three Years? Evaluation of the TASC After-School Program, December 2002, page 7, at 
http://www.tascorp.org/pages/psaYear3.pdf 1 

> “In general, the TASC participants who were at greatest academic risk made the largest math 
gains, when compared to other students. (Reporting of subgroup analyses focuses here on 
math because of the consistent relationships with TASC participation, as found in the 
aggregate analyses of math achievement.) Math benefits were clearly evident for students 
who scored in the lowest of four proficiency levels in the year prior to TASC participation. 
The gains for these low-achieving students were evident for active participants regardless of 
their number of years of participation. Among students from low-income families, the 
evaluation also found evidence of after-school benefits in math after two or more years of 
active participation.” [What Have We Learned, page 7.] 

> “Among the various subgroups examined, African-American students were especially likely 
to benefit from active participation in TASC projects, demonstrating gains in math over 
similar nonparticipants after one or more years of active participation. Hispanic students 
benefited in math after two years of participation.” [What Have We Learned, page 7.] 

Houston ’ s After-School Achievement Program (ASAP) 

In 1997, Houston’s After-School Achievement Program began providing significant funding for 
afterschool programs in the city. The program has grown steadily since, and in the 2000-2001 
academic year, it provided $2.3 million to 95 sites. ASAP has six programmatic goals: to 
reduce crime committed by and against juveniles; to prevent delinquency; to provide a safe, 
supervised place for youth; to provide academic enhancement and enrichment; to promote school 
attendance and discourage school drop-out; and to motivate youth to develop good citizenship. 

The program has been evaluated annually by independent evaluators, with the most recent study 
conducted by Dennis W. Smith, Ph.D. and James J. Zhang, P.E.D, covering the 2000-2001 
school year. Among their findings: 

> “In both science and fine arts, ASAP participants improved significantly over the course of 
the school years compared with students not in the ASAP. While student achievement in the 
remaining nine subject areas [reading, other language arts, mathematics, social studies. 
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handwriting, physical education, health and safety, computers and science lab] was not 
significantly different between the ASAP and non- ASAP groups, the post-test mean scores 
for ASAP participants clearly indicated improvement over the span of the program year.” 
[“Shaping our Children’s Future: Keeping a Promise in Houston Communities, 2001, Year 4 
Evaluation of the After-School Achievement Program,” page 4, published by ASAP. 
Contact: Jennifer Brimer, ASAP Coordinator, 713-437-6981.] 

> Surveys conducted for the evaluation found that 22 percent of parents of ASAP children said 
“their children would be by themselves” without ASAP, and “close to 16 percent of parents 
said that their children would be watched by a sibling.” [“Shaping,” page 6.] 

San Diego ’s ‘6 to 6’ Extended School Day Program 

San Diego has developed one of the nation’s most ambitious afterschool programs, with the goal 
of making affordable programs before and after school available to every elementary and middle 
school student in the City of San Diego. 

Two significant evaluations of the program have been conducted, one an interim report by 
WestED, released in April 2001, the other by Hoffman Clark and Associates released in July 
2001. 

Using random sampling of sites, document review, interviews, focus groups and site 
observations, WestED found: 

> Parents expressed high levels of satisfaction with the program, “including their perceptions 
of the quality of academic enrichment, the degree to which children looked forward to the 
program, communication with staff, success at helping children complete homework, and 
the promotion of positive behavior in children.” [Harvard Family Research Project website 
at http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/sd66esdp.pdf, hereafter 
HFRP-SD.] 

> “Almost two-thirds of responding parents noticed improvements in their children’s 
academic performance.” [HFRP-SD.] 

Relying on random sampling of program participants and reviewing a variety of data, WestED’s 
study concluded: 

> Reading scores for “6 to 6” students improved. Fifty-seven percent of students increased 
their reading scores over the course of the studied year, and SAT-9 reading scores 
increased. [HFRP-SD.] 

> Nearly ten percent of children moved up into the 25 th percentile or higher in 2000 by 
comparison to 1999 reading scores. [HFRP-SD.] 

> Forty-four percent of students increased their SAT-9 math scores. [HFRP-SD.] 

Ohio Urban School Initiative School Age Child Care Project 

The Urban School Initiative School Age Child Care Project (SACC) funds a variety of 
afterschool programs in Ohio urban school districts. The University of Cincinnati College of 
Education’s Evaluation Services Center conducted a thorough review of the program’s 1998- 
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1999 school year, measuring both project design and its outcomes. Data collection included 
document reviews, observation of programs, surveys and questionnaires. Among the findings: 

> “Ohio Proficiency Tests scores for both 4th and 6th graders showed that SACC children 
exceeded the state-wide percentages of students meeting proficiency standards. SACC 4th 
grade students’ scores exceeded the statewide percentages of students meeting proficiency 
standards in every subject area tested: writing, reading mathematics, citizenship, and 
science. SACC 6th graders exceeded the statewide percentages of students meeting 
proficiency standards in four of the five areas: writing, reading mathematics, and 
citizenship.” [Harvard Family Research Project at 

http://www.gse.harvard.edu/~hfrp/proiects/afterschool/mott/osisaccp.pdf , hereafter HFRP- 
Ohio.] 

> “School absence and tardiness were reduced for participating students. First graders who 
were not in a SACC program during kindergarten reduced the number of school days they 
missed from an average of 8 during their kindergarten year to an average of 3 days during 
their 1998-99 1st grade year. Eighth graders who were not in a SACC program during 7th 
grade reduced the average number of school days missed from 18 to 5.” [HFRP-Ohio.] 

> “Suspensions and expulsions, when comparing the 1 998-99 school year to the prior school 
year, were reduced for both elementary school students and middle school students who 
participated in SACC programs.” [HFRP-Ohio.] 

> “School buildings housing SACC programs were in use more hours of the day and weeks of 
the year because of these programs.” [HFRP-Ohio.] 

> “The adults in the participating children’s families had a greater awareness of community 
agencies, their facilities, and their services because of the SACC programs.” [HFRP-Ohio.] 

> “Parents participating in interviews or completing surveys felt the programs had positive 
impacts on their families.” [HFRP-Ohio.] 

> “Participating children spent more hours in a safe, supervised environment, before and/or 
after school, than they had prior to program involvement.” [HFRP-Ohio.] 

> “Participating children’s television and video viewing decreased because of attendance in 
this program.” [HFRP-Ohio.] 

YS-CARE After School Program for California Work Opportunity and Responsibility to Kids 

The YS-Care After School Program was established in 1999 by the Los Angeles County Board 
of Supervisors. The afterschool program is “designed to offer a safe environment that includes 
academic assistance, homework help, enrichment activities, recreation, and quality childcare 
provided by caring adults in well-supervised school site environments.” [. Evaluation of the YS- 
CARE After School Program For California Work Opportunity and Responsibility to Kids 
(CalWORKS), March 2002, at http://www.gse.uci.edu/asp/aspeval/resources/YSCARE13.pdf, 
page 5, hereafter YS-CARE Evaluation.] The program is targeted at K-5 children attending 
schools in neighborhoods with high concentrations of families receiving Temporary Assistance 
to Needy Families (TANF). The Department of Education, University of California at Irvine and 
Research Support Services conducted a formal evaluation of the program, and released findings 
in March 2002. 

The evaluation compared the test scores and behavior of participating students with a 
comparable group of non-participating students. The study’s chief conclusions: 
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> “YS-CARE participants had larger gains on SAT-9 Reading and SAT-9 Math scores than 
non-participants.” 

> “YS-CARE participants had larger gains on Reading Achievement than matched non- 
participants.” 

> “YS-CARE participants initially in the lowest decile reading group had significantly larger 
reading gains than matched non-participants.” 

> “YS-CARE participants had significantly lower scores on all Work and Study Habits and 
Citizenship measures at baseline. The participants narrowed the gap by the time of the end- 
of-year ratings, with almost half of the initial differences substantially smaller.” [YS-CARE 
Evaluation, pp. 5-6] 



Massachusetts After-School and Other Out-of-School Time Grant Programs 

In early 2003, the Massachusetts Department of Education released, via the Internet, a draft 
executive summary of an evaluation the state’s After-School and Other Out-of-School Time 
Grant Programs. The purpose of the program is “to establish or expand community learning 
centers that operate during out-of-school hours and provide students with academic enrichment 
opportunities along with other activities designed to complement the students’ regular academic 
program.” [http://www.doe.mass.edu/ose/asost/execsum drft.pdfl The evaluation report was 
submitted jointly by Beth Miller and Wendy Surr of the National Institute on Out of School Time 
at Wellesely College, and Karyl Resnick and Kelly Church of School Enrichment Services of the 
Massachusetts Department of Education. The report covers Fiscal Year 2002. 

According to the report, “Building an Outcome Evaluation System For the Massachusetts 
Department of Education’s 21 st Century Community Learning Centers,” FY 2002 saw important 
gains for students in a number of areas. According to the report: 

> “Results indicate that 56% of the students participating had positive gains on measured 
outcomes.” [“Building an Outcome Evaluation System,” 
http://www.doe.mass.edu/ose/asost/execsum drft.pdf .l 

> “Student gains in Math and/or English Language Arts were statistically significant in 73% 
of the ASOST programs.” 

> The areas with the greatest percentage of youth improving were Learning Skills, 
Communication Skills and Engagement in Learning. 



The Foundations After-school Enrichment Program 

For more than a decade, Foundations, Inc. has operated extended-day enrichment programs and 
provided technical assistance to other afterschool sponsors. During the 2001-2002 school year, 
Drs. Stephen P. Klein and Roger Bolus of Ganks & Associates (Santa Monica, Cal.) 
administered pre- and post-tests in mathematics and reading to first- through fifth-grade students 
in 19 Foundations programs in three states. A summary of the report is available on the 
Foundations website at http://www.foundationsinc.org/ExtendedDayFolder/conclusions.asp . 

The full report, issued in December 2002, concludes: 
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> “Foundations students made substantial improvements in average scores between the fall 
pretest and spring posttest. In fact, their average score gains in mathematics were somewhat 
greater than what would be expected given the results obtained in CTB/McGraw-Hill’s national 
norm sample. The Foundations students’ gains in reading kept pace with those made in this 
national norm sample.” [“Improvements in Math and Reading Scores of Students who Did and 
Did Not Participate in the Foundations After School Enrichment Program During the 2001-2002 
School Year,” Stephen P. Klein, Ph.D. and Roger Bolus, Ph.D., Gansk & Associates, December 
2, 2002, page 2.] 

> Over the course of the school year, Foundations’ afterschool students’ test averages moved 
them up national percentile rankings - by an average of 10 percentile rankings in mathematics 
reading, and an average of 2 percentile rankings in reading. [“Improvements in Math and 
Reading Scores,” page 10.] 

> Foundations students fared very well by comparison to non-Foundations students at the 
studied schools. The mathematics “effect size” difference averaged .39 (representing 39/100 ths 
of a standard deviation unit), in afterschool students’ favor. In reading, a similar finding: a .41 
effect size advantage for afterschool students. [“Improvements in Math and Reading Scores,” 
page 14.] 

The Extended-Service Schools Initiative: 2002 Report 

In 1998, the Extended-Service Schools Initiative (ESS) began funding community organizations 
across the country to partner with local schools to create a total of 60 afterschool programs in 20 
communities. Each of the programs follows one of four nationally recognized program models - 
Beacon, Bridges to Success, Community Schools, or West Philadelphia Improvement 
Corporation - to provide youth-development activities in low-income areas in programs located 
in school buildings during non-school hours. ESS was funded as a five-year program by 
Wallace-Reader’s Digest Funds. Separately, the Wallace-Reader’s Digest Funds provided 
financial support to Public/Private Ventures (PPV) to conduct an evaluation of the program. 

PPV, with subcontractor Manpower Demonstration Research Corporation, began a four- year, 
multi-phase evaluation. Issued in June 2002, Multiple Choices: Findings from the Extended- 
Service Schools Initiative , is available online at http://www.ppv.org/content/reports/ess-multi- 
full.html . 

According to researchers, “Students who participated in the school-based, afterschool programs 
seemed to experience positive change in four key areas: staying out of trouble; improving their 
school attitudes and behavior; strengthening their social networks; and learning new skills, 
seeing new possibilities and improving their self-confidence.” [. Multiple Choices, page 30, 
http://www.ppv.Org/content/reports/ess-multi-full.html .1 Specifically: 

• “One particularly important outcome desired for after-school programs is that they decrease 
the risktaking behavior of youth. By providing them with structured, supervised activities, 
the time they have to get into trouble is decreased. In addition, the social rules and tone 
implemented by staff can teach youth to deal more appropriately with negotiation, social 
conflict and anger. When we asked parents and youth if ESS helped them stay out of trouble 
and more appropriately deal with conflict, both groups — but especially the parents — 
believed ESS was very useful in this regard.” [. Multiple Choices, page 31 .] 



• “Given that most of the ESS programs were new and the levels of participation were well 
below five days a week, it was not thought likely that we would observe changes in grades or 
test scores. However, to gauge whether ESS was starting to have positive academic effects, 
we asked parents and youth if they thought the program helped the youth do better in school. 
In addition, we measured some “leading indicators” of academic improvement (a sense of 
academic mastery and the level of school effort) to ensure that we did not miss an important 
change if one had occurred. As Table 7 illustrates, approximately two-thirds of the youth 
believed the program helped them do better in school, and it was even more likely that the 
parents found the program helpful to their children in this way.” [Multiple Choices, page 32.] 

• “Interestingly, the parents’ survey responses are consistent with the expected pathway of 
change that could ultimately lead to increased academic success. High percentages of parents 
felt that ESS helped their children like school more and try harder in school, factors that may 
lead to learning more and doing better.” [. Multiple Choices, page 32.] 

• “When we examined how the youth’s academic attitudes and behaviors changed over time, 
we found a consistent story. Youth who participated in ESS activities experienced a greater 
increase in their sense of belonging at school and paid more attention in class. Again, 
consider the two groups of similar youth. ... [A]mong the youth who did not go to ESS 
during the 13 months between the initial and follow-up surveys, 20 out of 100 reported that 
they started skipping school, 29 said they really paid attention in class, and 76 said they were 
very proud to belong to their school. Among similar youth who went to ESS two days a 
week, only 1 1 out of 100 reported starting to skip school; 49 said they really paid attention in 
class; and 84 said they were very proud to belong to their school.” [ Multiple Choices, page 
32.] 

• “Responses on the parent survey administered in Spring 2001 suggest that the after-school 
programs were having some of these beneficial outcomes: 80 percent of parents said they 
were less worried about their child’s safety after school. 57 percent said their child’s 
participation helped them manage their own work schedule. 47 percent said it let them attend 
classes or job training more easily. 45 percent said it helped them get a better job or do better 
at their job.” [Multiple Choices, pages 33-34.] 

* * * * 

Two other recent evaluations offer important insights into the barriers afterschool programs face. 
The MOST Initiative: Making the Most of Out-of-School Time 

The MOST Initiative is a multi-year project of the National Institute on Out-of-School Time 
(NIOST) at the Wellesley Centers for Women, funded by Wallace-Reader’s Digest Funds. The 
project is aimed at improving the quality and availability of out-of-school-time programming for 
children, particularly low-income children, in three major American cities - Boston, Chicago and 
Seattle. In addition to funding the project, Wallace-Reader’s Digest Funds also funded a 
comprehensive evaluation of the project, conducted by the Chapin Hall Center for Children at 
the University of Chicago. A summary of the findings of the two-phase evaluation is available at 
http://www.welleslev.edu/WCW/CRW/SAC/most.html . Its findings address, among other 
things, the need for adequate funding for afterschool programs: 
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> The first phase of the evaluation, completed in 1997, found that MOST contributed to the 
supply of school-age care in the cities, making “moderate to substantial progress toward the 
goal of increasing the supply of slots.” 

> In the final evaluation, completed and released in 2000, researchers noted the wide range of 
afterschool providers in the three cities, and observed that private nonprofits are the largest 
providers. They observed further that just a small share of the cities’ low-income children 
were in afterschool programs, and concluded that inadequate public financing of afterschool 
programs led to an insufficiency of slots and depressed parental demand. The study also 
cited as barriers transportation difficulties, a lack of parental awareness of afterschool 
programming, and the lack of parental wherewithal to seek out and enroll children in 
programs. Finally, the shortage of public financing also was found to create staffing and 
programmatic problems that hurt the overall quality of afterschool programming. 

(Under the leadership of Mayor Thomas Menino, Boston has since launched a large-scale public- 
private partnership to significantly increase its afterschool funding.) 

The Extended-Service Schools Initiative: 2000 Report 

As noted earlier, the Extended Services Schools Adaptation Initiative (ESS) funds community 
organizations across the country to partner with local schools to create and support 60 
afterschool programs in 20 communities. The Wallace-Reader’s Digest Funds provided funding 
for ESS, and for a separate evaluation of the program, conducted by Public/Private Ventures 
(PPV) and subcontractor Manpower Demonstration Research Corporation. The 2002 report 
from PPV examined a number of topics, including student outcomes, covered earlier in this 
backgrounder. Earlier reports from PPV explored a number of programmatic issues, casting 
light on the range of challenges afterschool programs confront. Several of those discussed in 
PPV’s 2000 report on ESS are highlighted below. 

> Issues related to physical space in the school building can have a profound impact on the 
success of the program. Afterschool programs must sometimes compete for facilities with 
other activities and events. Relationships with school personnel, including administrators 
and janitorial staff can be critical. Principals sometimes deny access to already overused 
and underfunded facilities. [Extended Services in Schools: Putting Programs in Place, 
http://www.ppv.org/pdffiles/ess.pdf, pages 34-39.] 

> The most disadvantaged children are very difficult to reach. ESS programs reach thousands 
of disadvantaged children, but within poor neighborhoods, the poorest children are the most 
difficult to reach and involve in afterschool. The programs have worked to develop creative 
strategies for overcoming this barrier. [Putting Programs in Place, page 27.] 

> Transportation is the most difficult challenge facing the programs. Limited availability of 
buses, unsafe neighborhoods, and distances between schools and homes all combine to 
make transportation a difficult problem. [Putting Programs in Place, page 40-42.] 

The Afterschool Alliance is a nonprofit public awareness and advocacy organization supported 
by a group of public, private, and nonprofit entities dedicated to ensuring that all children and 
youth have access to afterschool programs by 2010. The Alliance is proud to count among its 
founding partners the C.S. Mott Foundation, U.S. Department of Education, JCPenney 
Afterschool, Open Society Institute/The After-School Corporation, the Entertainment Industry 
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Foundation and the Creative Artists Agency Foundation. Washington, DC Office: 202/347- 
2030; Flint, Michigan Office: 810/239-3449 
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